Video output device and video output method

ABSTRACT

A video output device includes a screen configuration management section for managing a placement or a scaling of a screen for displaying at least one video, an area determination section for determining a character area in an input video, a supplemental information storing section for cutting out a video on a supplemental information area including the at least one character area determined by the area determination section and storing it as supplemental information, a supplemental information display determination section for determining whether or not the supplemental information stored by the supplemental information storing section is displayed in addition to an input video based on the screen configuration managed by the screen configuration management section, and an output section for outputting the input video and the supplemental information in accordance with commands from the screen configuration management section and the supplemental information display determination section. The supplemental information display determination section determines that the supplemental information is displayed based on a size of a character included in the character area in the input video changing in response to a scaling managed by the screen configuration management section. Therefore, the display form of the supplemental information can be changed in response to a scaling factor of the display size of the input video.

TECHNICAL FIELD

This invention relates to a video output device and a video outputmethod for changing the display form of supplemental information inresponse to the scaling factor of the display size of input video.

BACKGROUND ART

Generally, a device for receiving a broadcast program provided by a TVbroadcast station, a cable TV station, etc., processes a receptionsignal and outputs video. Hereinafter, the device will be called “videooutput device.” The video output destination is a display installed inthe video output device or an external display connected to the videooutput device through a communication member of a cable, etc. At thistime, to meet the demand of the user who wants to view more than oneprogram at the same time, for example, “two-screen mode” of displayingtwo input video images on two divided screens is widely known.

FIGS. 1( a) to (c) are drawings to show display patterns of generaltwo-screen mode. FIG. 1( a) shows an example of displaying input video Aand input video B in parallel in the same ratio. FIG. 1( b) shows anexample of displaying input video A and input video B in differentsizes. FIG. 1( c) shows an example of superposing input video B of asmall screen on input video A displayed on the full screen. FIGS. 2( a)to (c) show display examples of assigning video to each display patternof the two-screen mode shown in FIGS. 1( a) to (c).

When video is displayed on a display in the two-screen mode, generallythe user performs predetermined operation, thereby switching the screendivision pattern at a fixed ratio. However, recently, a video outputdevice for dividing a screen at any desired ratio specified by the userhas also appeared.

On the other hand, in a recent video output device, the screen becomeslarger and higher functions are provided. For example, a modelinstalling a plurality of tuners, a model installing a plurality ofexternal input terminals that can be connected to a gaming machine,etc., a model of playing back a moving image received through theInternet, a model of recording a received program on a record medium forstorage, and the like are available. Consequently, one large screen in aliving room may be divided into three parts and the family may viewthree video images. For example, the following use is assumed: A motherand a daughter enjoy a recorded drama on a first division screen, afather enjoys a baseball relay on a second division screen, and a sonenjoys a game on a third screen.

If a plurality of video images are scaled down and displayed at a fixedratio or any desired ratio on the display of the video output devicehaving the screen dividing function described above, a character stringor a symbol in the scaled-down video may be crushed as shown in FIG. 2(a) to (c). At this time, the displayed information is hard to read bythe user and cannot be recognized by the user; this is a problem.

One method of solving this problem is disclosed in Patent Document 1.According to the method in the document, when a program involving databroadcast is scaled down for display by screen division, the usersuperposes a menu screen of data broadcast specified by the programsending party outside the scaled-down screen based on layout informationby remote control operation.

However, this method assumes that a predetermined character string andimage data are involved in the program as data broadcast together withlayout information. The user must continuously operate the remotecontroller to display information to be checked. Since information issuperposed on a different division screen, the different division screenis hidden by the superposition screen and information is lost.

Another method to solve the problem is disclosed in Patent Document 2.In the method in the document, for two or more input video images, afirst input video is displayed on the full screen and a subtitle areacontaining a character string is extracted from the second input videoand the extracted character string is superposed on any desired positionof the first input video.

FIGS. 3 to 5 are drawings to show an outline of processing according tothe method disclosed in Patent Document 2. In the method in PatentDocument 2, of two input video images A and B shown in FIG. 3( a), acharacter string B′ is extracted from the input video B as shown in FIG.3( b) and is superposed on the input video A as shown in FIG. 3( c).FIG. 4 shows a display example when video is assigned to each screen. Inthe example shown in FIG. 4, the score display portion corresponding tothe character string area B in the baseball relay corresponding to theinput video B is superposed on the input video A.

Related Art Documents Patent Documents

Patent Document 1: JP-A-2002-171457

-   Patent Document 2: JP-A-8-317301

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

In the method disclosed in Patent Document 2 described above,information to be known by the user in the input video needs to be acharacter string. Thus, information which is not a character string (forexample, logo mark of professional baseball team, etc.,) is notcontained in the character string area B′. Therefore, if information tobe known by the user contains information which is not a characterstring, information is lost (cut); this is a problem. For example, if asymbol or a mark that cannot be extracted is used in the score displayportion of the input video B shown in FIG. 4 as shown in FIG. 5( a),only the character string portion is extracted as shown in FIG. 5( b).Thus, a symbol and a mark are not superposed on the input video A asshown in FIG. 5( c).

Although the character string area B′ can be superposed on any desiredposition of the input video A, since the input video A is full screendisplay, the character string area B′ is superposed on any location ofthe input video A. Thus, partial information of the input video A islost. For example, as shown in FIGS. 4( c) and 5(c), the characterstring area B′ is superposed on a part of a telop of the input video Aand thus information of the telop in the superposition portion is lost.

It is an object of the invention to provide a video output device and avideo output method for changing the display form of supplementalinformation in response to the scaling factor of the display size ofinput video.

Means for Solving the Problems

The invention provides a video output device including a screenconfiguration management section that manages a placement or a scalingof a screen for displaying at least one video; an area determinationsection that determines a character area in an input video; asupplemental information storing section that cuts out a video on asupplemental information area including at least one character areadetermined by the area determination section and stores the video on thesupplemental information area as supplemental information; asupplemental information display determination section that determineswhether or not the supplemental information stored by the supplementalinformation storing section is displayed in addition to an input videobased on a screen configuration managed by the screen configurationmanagement section; and an output section that outputs the input videoand the supplemental information in accordance with commands from thescreen configuration management section and the supplemental informationdisplay determination section. The supplemental information displaydetermination section determines whether the supplemental information isdisplayed based on a size of a character included in the character areain the input video which changes in response to the scaling managed bythe screen configuration management section.

The invention provides a video output device including a screenconfiguration management section that manages a placement or a scalingof a screen for displaying at least one video; an area determinationsection that determines a character area in an input video; acommunication section that acquires a character string included in atleast one character area determined by the area determination section assupplemental information from an external server; a supplementalinformation display determination section that determines whether or notthe supplemental information acquired by the communication section isdisplayed in addition to an input video based on the screenconfiguration managed by the screen configuration management section;and an output section that outputs the input video and the supplementalinformation in accordance with commands from the screen configurationmanagement section and the supplemental information displaydetermination section. When a size of a character included in the atleast one character area determined by the area determination section issmaller than a threshold value, the supplemental information displaydetermination section displays the supplemental information.

The invention provides a video output method including: determining acharacter area in an input video; cutting out a video on a supplementalinformation area including at least one character area and storing thevideo on the supplemental information area as supplemental information;determining whether or not to display the supplemental information inaddition to an input video based on the screen configuration fordefining a placement or a scaling of a screen for displaying at leastone video; and outputting the input video and the supplementalinformation in accordance with the screen configuration and thedetermination result. It is determined that the supplemental informationis displayed based on a size of a character included in the characterarea in the input video.

The invention provides a video output method including: determining acharacter area in an input video; acquiring a character string includedin at least one character area as supplemental information from anexternal server; determining whether or not to display the supplementalinformation acquired from the external server in addition to an inputvideo based on the screen configuration for defining a placement or ascaling of a screen for displaying at least one video; and outputtingthe input video and the supplemental information in accordance with thescreen configuration and the determination result. When a size of acharacter included in the at least one character area is smaller than athreshold value, it is determined that the supplemental information isdisplayed.

Advantages of the Invention

According to the video output device and the video output methodaccording to the invention, the display form of supplemental informationcan be changed in response to the scaling factor of the display size ofinput video. Thus, if screen division of a plurality of input videoimages is performed, the portion where crush of a character or a symbolin input video occurs can be complemented by supplemental informationand loss of information that the user wants to know can be prevented.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1( a) to (c) are drawings to show display patterns of generaltwo-screen mode.

FIG. 2( a) to (c) are drawings to show display examples of assigningvideo to each display pattern in the two-screen mode shown in FIGS. 1(a) to (c).

FIG. 3 is a drawing to show a display example of cutting out andsuperposing a part of input video B according to a method disclosed inPatent Document 2.

FIG. 4 is a drawing to show a display example of assigning video to eachscreen shown in FIG. 3.

FIG. 5 is a drawing to show a display example wherein input video Bcontains a character string and a symbol.

FIG. 6 is a block diagram to show a video output device of a firstembodiment and its relevant devices.

FIGS. 7( a) to (f) are drawings to show patterns of full screen displayand two-screen display according to the first embodiment.

FIG. 8 is a drawing to show an example of a screen configurationmanagement table held by a screen configuration management section 103.

FIGS. 9( a) to (f) are drawings to show data configuration examples ofthe screen configuration management table held by the screenconfiguration management section 103.

FIG. 10 is a drawing to show a display example of a supplementalinformation screen in two-screen display state C shown in FIG. 7( e).

FIG. 11 is a drawing to describe screen elements shown in FIG. 10.

FIG. 12 is a flowchart to show processing performed by an image outputdevice 100 in displaying a supplemental information screen in responseto a screen definition request.

FIG. 13 is a flowchart to show supplemental information extractionprocessing.

FIG. 14 is a drawing to show a character area and an object area ininput video 2.

FIG. 15 is a drawing to show a character area table held by asupplemental information storing section 113.

FIG. 16 is a drawing to show an object area table held by thesupplemental information storing section 113.

FIG. 17 is a flowchart to show character area group definitionprocessing shown at step 213 in the flowchart of FIG. 13.

FIG. 18 is a drawing to show the superposition state of a character areaand an object area in input video 2.

FIG. 19 is a drawing to show a character area group table held by thesupplemental information storing section 113.

FIG. 20 is a drawing to a superposition state detection pattern.

FIG. 21 is a drawing to show an example of a margin width used ininternal processing.

FIG. 22 is a flowchart to show the supplemental information definitionprocessing shown at step S214 in the flowchart of FIG. 13.

FIG. 23 is a drawing to show definition of character area group area andsupplemental information area in input video 2.

FIG. 24 is a drawing to show a supplemental information table held bythe supplemental information storing section 113.

FIG. 25 is a drawing to show supplemental information displaydetermination criterion responsive to the scaling factor and the screendivision mode of input video.

FIG. 26 is a drawing to show an example of a factor for determining thedisplay location of supplemental information.

FIG. 27 is a drawing to show the screen layout configuration of“two-screen display state C” shown in FIG. 7( e).

FIG. 28 is a block diagram to show the video output device of the secondembodiment and its relevant devices.

FIG. 29 is a drawing to show screen configuration data provided by asupplemental information providing device.

FIG. 30 is a drawing to show a display example of supplementalinformation based on the screen configuration data.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of a video output device and a video output method accordingto the invention will be discussed below with reference to theaccompanying drawings: The video output device of each embodimentdescribed below is a device for receiving content of a program,commercials, etc., sent through a sending station by a broadcast stationand displaying the content on a display inside or outside the device.

First Embodiment

FIG. 6 is a block diagram to show a video output device of a firstembodiment and its relevant devices. To begin with, a processing outlineuntil the video output device displays input video on a display will bediscussed.

As shown in FIG. 6, an image output device 100 includes an input section101, a terminal control section 102, a screen configuration managementsection 103, a video input section 104, an application processingsection 105, a screen generation section 106, a combining section 107,an output section 108, an area determination section 112, a supplementalinformation storing section 113, and a supplemental information displaydetermination section 115.

The input section 101 receives a control instruction signal from aremote control device (simply called “remote controller”) attached tothe image output device 100 or an external connection device, etc., andtransfers the signal to the terminal control section 102. The terminalcontrol section 102 sends an inquiry of the screen configurationresponsive to the control instruction signal transferred from the inputsection 101 to the screen configuration management section 103. Thescreen configuration is, for example, the configuration of displayingthe last viewed channel on the full screen in accordance with a power oninstruction of the image output device 100, the configuration of scalingdown the current broadcast for display and displaying a program guide onthe full screen in accordance with a program guide display instruction,etc. The screen configuration management section 103 manages theconfiguration of a screen displayed on the display. The screenconfiguration management section 103 stores a screen configurationmanagement table described later.

When a program guide display instruction signal is input to the inputsection 101, the terminal control section 102 transfers the programguide display instruction to the application processing section 105. Theapplication processing section 105 can execute applications forrealizing various functions in addition to a program guide displayfunction. The application processing section 105 returns processingrequired for the terminal control section 102 (for example, scaling downof the current channel for display, etc.,) in response to theinstruction input from the terminal control section 102. The applicationprocessing section 105 provides information required for generating ascreen required for setting responsive to the instruction from theterminal control section 102 for the screen generation section 106. Thefunctions realized by applications that can be executed by theapplication processing section 105 are not only the program guidedisplay function, but also a broadcast reception setting function, animage quality adjustment function, an alarm setting function, a calendardisplay function, a network connection setting function, etc., forexample.

The screen generation section 106 performs drawing processing of basicparts (character string, image, background color, etc.,) combined into ascreen based on the information provided from the application processingsection 105.

The video input section 104 manages at least one of video sources thatcan be input by the image output device 100 and transfers input videoobtained in response to a command from the terminal control section 102to the combining section 107. In the embodiment, the video input section104 has, for example, a first tuner 109 a, a second tuner 109 b, anexternal input section 109 c, a communication section 109 d, decoders110 a to 110 d, and processing sections 111 a to 111 d.

The first tuner 109 a and the second tuner 109 b receive content of aprogram, commercials, etc., sent through a sending station 400 by abroadcast station 300 through an antenna 200. The external input section109 c acquires content output from a video providing device 500 of arecorder having a record medium or any other video device, etc. Further,the communication section 109 d receives content transmitted from avideo providing device 700 through a network 600. The decoders 110 a to110 d decode signals from the first tuner 109 a, the second tuner 109 b,the external input section 109 c, and the communication section 109 drespectively. The processing sections 111 a to 111 d perform processingof changing a cast of video, the scaling factor, etc., in response to acommand from the terminal control section 102 using video data from thecorresponding decoders 110 a to 110 d and transfer to the combiningsection 107.

The combining section 107 combines video transferred from the videoinput section 104 based on setting of the screen configurationmanagement section 103 and output the combined video (which will behereinafter called “resultant video”) to the output section 108.Specifically, the combining section 107 combines each input video scaleddown at the scaling factor required from the terminal control section102 by the processing sections 111 a to 111 d in the screenconfiguration based on the layout specified for the screen configurationmanagement section 103 from the application processing section 105.

If the image output device 100 includes a display, the output section108 displays the resultant video on the display; if the image outputdevice 100 does not include a display, the output section 108 transfersthe resultant video to an external display.

The area determination section 112, the supplemental information storingsection 113, and the supplemental information display determinationsection 115 included in the image output device 100 will be discussed indetail based on a specific example of extraction processing and displayprocessing of supplemental information. FIGS. 7( a) to (f) are drawingsto show patterns of full screen and two-screen display according to thefirst embodiment. FIG. 8 is a drawing to show an example of a screenconfiguration management table held by the screen configurationmanagement section 103.

FIG. 7( a) shows a state in which input video 1 received by the firsttuner 109 a is displayed on the full screen of the display as theinitial state (full screen display state of only a drama program). FIG.7( b) shows a state in which input video 1 received by channel change ofthe first tuner 109 a is displayed on the full screen of the display bychannel change when the user gives a channel change command using theremote controller (full screen display state of only a baseball relay).

FIG. 7( c) shows a state in which when the user gives a two-screendisplay command using the remote controller, the previously selectedchannel (drama) as input video 1 and the current selected channel(baseball relay) as input video 2 are displayed at the screen ratio ofone to three (two-screen display state A).

FIG. 7( d) shows a display example in which when the user can change theleft-right screen ratio as desired operating a left-right button of theremote controller, the user increases the ratio of the left screen andsets the screen ratio of the left screen and the right screen to thesame ratio. In the display screen, it is determined that the user cannotdetermine the batter score (the character string and the symbol) of thebaseball relay of the right screen (input video 2) in the divisionscreen. Thus, in the display example, the screen of the area indicatingthe batter score of the right screen (input video 2) is displayed in theactual size as supplemental information in the proximity of the rightscreen (input video 2) (two-screen display state B).

FIG. 7( e) shows a display example in which the ratio of the left screenis set large. In the display example, it is determined that the usercannot determine progress information (character string and symboldisplayed in the lower right portion of the screen) of the baseballrelay of the right screen (input video 2) in the division screen. Thus,in the display example, in addition to a screen of an area indicatingthe batter score, a screen of an area indicating the progressinformation of the right screen (input video 2) is also displayed in theactual size as supplemental information in the proximity of the rightscreen (input video 2) (two-screen display state C).

FIG. 7( f) shows a display example in which the ratio of the left screenis maximized. In the display example, the baseball relay of the rightscreen (input video 2) is minimized to the level such that the generalsituation in which the batter swings, etc., can be determined. However,the screens of the areas indicating the batter score and the progressinformation are displayed in the actual size as they are superposed on apart of the left screen as supplemental information (input video 1)(two-screen display state D).

In the description, the previously selected channel is input video 1 andthe current selected channel is input video 2. However, for selection ofinput video for the channel, the current selected channel may be inputvideo 1 received by the first tuner 109 a and the last channel selectedby the user in the second tuner 109 b may be input video 2. The user mayselect either the left or the right for change.

The screen ratio when a two-screen display command is given is notlimited to one to three and may be set to any ratio. In the description,the left and right screen ratio can be changed as desired operating theleft-right button of the remote controller, but any input means such asanother button or a touch sensor or an acceleration sensor installed inthe remote controller may be used. Further, FIG. 7( f) shows an examplein which the screens of the areas indicating the batter score and theprogress information of supplemental information are superposed on apart of the left screen (input video 1). However, enlarging of the leftscreen may be controlled so that the screen indicating the supplementalinformation is not superposed on the left screen. When the screenindicating the supplemental information is superposed on the leftscreen, the transparency of the screen indicating the supplementalinformation may be lowered. At this time, the user can also check inputvideo 1 of the overlap portion of the screen indicating the supplementalinformation.

FIG. 8 shows a data configuration example of the screen configurationmanagement table held by the screen configuration management section 103in the state shown in FIG. 7( a). As shown in FIG. 8, an input source,an output flag, a width (W) and a height (H) indicating the drawing areaof a screen, a drawing position of a horizontal axis (X), a drawingposition of a vertical axis (Y), depth indicating a superposition layerindicating overlap of video images (Z), transparency of the screenindicating in the range of 0 to 255 (α), and scaling factor (R) relativeto a full size are set in the screen configuration management table foreach input video. As the transparency (α) is closer to 0, transparencyis higher.

The state shown in FIG. 7( a) is a state in which only input video 1received by the first tuner 109 a is displayed. Therefore, in the screenconfiguration management table at this time, only the output flag ofinput video 1 is “true” and the output flag of any other input video is“false.” As the drawing area of input video 1, the width (W) is 1920,the height (H) is 1080, the drawing position of the horizontal axis (X)is 0, and the drawing position of the vertical axis (Y) is 0. Further,as the parameters indicating overlap of input video 1 and any othervideo, the depth (Z) is 0, the transparency (α) of input video 1 is 255and the scaling factor (R) is 100%. When such parameters are set, inputvideo 1 is displayed on the full screen of the display.

FIGS. 9( a) to 9(f) show data configuration examples of the screenconfiguration management table corresponding to the display states inFIGS. 7( a) to 7(f). As shown in FIGS. 9( a) to 9(f), for the parametersof the screen configuration management table, if the screenconfiguration varies, change in the output flag, the screenconfiguration ratio, and the scaling varies.

Next, a procedure for displaying screens indicating supplementalinformation on the display will be discussed with reference to FIGS. 10to 12. FIG. 10 is a drawing to show a display example of a supplementalinformation screen in the two-screen display state C shown in FIG. 7(e). FIG. 11 is a drawing to describe screen elements shown in FIG. 10.

A display screen (1000/1100) is made up of input video images of twosystems. A left screen (1001) is screen 1 (1101) for displaying inputvideo 1 and a right screen (1002) is screen 2 (1102) for displayinginput video 2. An area (1003) indicating the batter score of input video2 (baseball relay) is displayed on screen 2 (1102) as input video 2 isscaled down, and thus it is determined that the area is an illegiblearea for the user, and the area is set to a supplemental informationtarget area SA1. Likewise, it is determined that an area (1004)indicating progress information of input video 2 (baseball relay) is anillegible area for the user, and the area is set to a supplementalinformation target area SA2.

In FIGS. 10 and 11, the portion provided by cutting out and displayingthe supplemental information target area SA1 is supplemental informationscreen 1 (1005/1105), and the portion provided by cutting out anddisplaying the supplemental information target area SA2 is supplementalinformation screen 2 (1006/1106). Further, as “enhancementrepresentation section” (1007/1107) indicating a supplementalinformation screen, shadow (drop shadow) displayed on the outer frame ofsupplemental information screen 2 (1006) or a symbol such as an asteriskdisplayed on the left of supplemental information screen 2 (1006). InFIGS. 10 and 11, dotted lines are displayed from the supplementalinformation target area SA1 to supplemental information screen 1 andfrom the supplemental information target area SA2 to supplementalinformation screen 2. The dotted lines are shown as a guideline toindicate “which area the information is displayed in on an enlargedscale” for the user. The dotted line as a guideline is shown by way ofexample; it may be an arrow, a solid line, a color, light and shade,etc., if which area is indicated can be determined.

Next, processing performed by the image output device 100 for displayingthe screens in FIGS. 10 and 11 on the display will be discussed indetail as “screen division mode processing,” “supplemental informationextraction processing,” “character area group definition processing,”and “supplemental information definition processing” separately.

FIG. 12 is a flowchart to show processing performed by the image outputdevice 100 in displaying a supplemental information screen in responseto a screen definition request. First, when the user presses a “screendivision mode button” of the remote controller, the input section 101receives an instruction signal of a “screen division mode request” fromthe remote controller and transfers the instruction to the screenconfiguration management section 103 through the terminal controlsection 102. The screen configuration management section 103 sets thenumber of screen divisions (N) to “2” and determines the screen ratio atthe display time (for example, fixed scaling factor 1:3 as the initialvalue) as initialization processing of the screen division modeprocessing. Further, the screen configuration management section 103instructs the video input section 104 to output input video 1 and inputvideo 2 from the first tuner 109 a and the second tuner 109 b throughthe terminal control section 102. The screen configuration managementsection 103 instructs the processing sections 111 a and 111 b to scaledown each input video so as to become the fixed scaling factor (1:3)through the terminal control section 102 (step S101).

In the description, the initial value of the screen ratio of screendivision is set to the fixed scaling factor (1:3), but may be set to thefixed scaling factor (1:1) or the user may previously specify anydesired scaling factor. Further, in the description, the number ofscreen divisions (N) is set to “2,” but may be set to more than 2 andthe screen ratio can be set as desired.

If the user again presses the “screen division mode button” withoutwaiting for display during processing at step S101, the screenconfiguration management section 103 terminates the screen division modeprocessing (Y at step S102). On the other hand, if the screen divisionmode is continued, the screen configuration management section 103continues processing (N at step S102).

Next, a flow to “supplemental information extraction processing” will bediscussed. At step S101, the number of screen divisions (N) is set to“2” and thus the supplemental information extraction processing isstarted at input image 2 indicated as N=2. First, a signal received bythe second tuner 109 b of the video input section 101 through theantenna 200 is decoded by the decoder 110 b. Then, the decoder 110 btransfers input video provided by decoding to the area determinationsection 112. The area determination section 112 executes supplementalinformation extraction processing for input video transferred from thedecoder 110 b (step S103). The details of the processing will bediscussed later.

If a character area to be set to supplementary information exists andsupplementary information is extracted as the result of the supplementalinformation extraction processing, the area determination section 112stores the extracted supplementary information in the supplementalinformation storing section 113. If input video contains supplementalinformation (Y at step S104) and a predetermined determination conditionis satisfied (Y at step S105), the supplemental information displaydetermination section 115 instructs the screen configuration managementsection 103 to superpose the supplemental information on the specifiedposition on the input video (highest layer) (step S106). According tothe instruction, supplemental information screen 1 (1005) andsupplemental information screen 2 (1006) shown in FIG. 10 are displayedon the display. The determination condition used at step S105 is“occurrence frequency of supplemental information is equal to or morethan predetermined occurrence frequency threshold value (for example50%),” “character size of extracted supplemental information×currentinput video scaling factor is equal to less than predetermined charactersize threshold value (for example, 12 points),” etc.

Next, if a change occurs in the previously held supplemental informationand the current extracted supplemental information as a history of thesupplemental information stored in the supplemental information storingsection 113, the supplemental information display determination section115 changes a change flag to status of “true.” If the change flag is“true” (Y at step S107), the supplemental information displaydetermination section 115 additionally displays enhancementrepresentation to the displayed supplemental information (step S108).That is, as the enhancement representation of the supplementalinformation display determination section 115, shade of the outer frameof supplemental information screen 2 (1006) shown in FIG. 10 or a symbol(1007) of an asterisk, etc., is displayed. On the other hand, if thechange flag is “false,” the supplemental information displaydetermination section 115 goes to the next step without displayingenhancement representation (N at step S107).

Further, the supplemental information display determination section 115manages “guideline display mode” as setting that can be changed by theuser. If the guideline display mode is set (Y at step S109), thesupplemental information display determination section 115 displays aguide line connecting the supplemental information target area of inputvideo and the supplemental information screen by dotted lines (stepS110). On the other hand, if the guideline display mode is not set, thesupplemental information display determination section 115 does notdisplay a guideline and goes to the next step (N at step S019).

In the embodiment, extraction of supplemental information 1 andsupplemental information 2, enhancement representation involved insupplemental information screen 2, and determination processing ofcomponent of guideline display have been described. To simplify thedescription, the display determination processing of supplementalinformation 1 and supplemental information 2 is described at the sametime; however, the display determination processing for supplementalinformation 1 and supplemental information 2 is performed in sequence asshown at steps S111 and S112 of the flowchart of FIG. 12. Further, thedisplay determination processing of input video 1 is performed followingthe display determination processing of input video 2 as shown at stepsS113 and S114. The order of the display determination processing dependson implementation. For example, at step S114, if the displaydetermination processing for input video 1 is not executed (N at stepS114), the process returns to step S103 for input video 1. Since anappropriate character area does not exist as supplemental information ininput video 1 shown in FIG. 10, if step S103 is executed, supplementalinformation is not extracted and thus it is determined at step S104 thatsupplemental information does not exist (N at step S104). If the displaydetermination processing for input video 1 and input video 2 is complete(Y at step S114), the screen 1000 containing the supplementalinformation screen shown in FIG. 10 is displayed on the display duringpredetermined wait time (Tw seconds) (step S115).

After the termination of the wait processing at step S115, if the screendivision mode continues (N at step S102), the video output device 100repeats steps S103 to S115 to update the supplemental information. Onthe other hand, if the screen division mode terminates (Y at step S102),the screen configuration management section 103 displays any input video(for example, input video 1) on the full screen. Thus, the screenconfiguration management section 103 instructs the processing section111 a of the video input section 104 to change the scaling factor andinstructs the combining section 107 to change the screen configurationthrough the terminal control section 102. Consequently, the screen shownin FIG. 7( a) is displayed on the display (step S116).

Next, “supplemental information extraction processing” of the areadetermination section 112 shown at step S103 in FIG. 12 will bediscussed in detail with reference to FIGS. 13 to 15 and FIG. 18. FIG.13 is a flowchart to show the supplemental information extractionprocessing. FIG. 14 is a drawing to show a character area and an objectarea in input video 2. FIG. 15 is a drawing to show a character areatable held by the supplemental information storing section 113. FIG. 16is a drawing to show an object area table held by the supplementalinformation storing section 113. FIG. 18 is a drawing to show thesuperposition state of a character area and an object area in inputvideo 2.

Preprocessing of the supplemental information extraction processingshown at step S103 in FIG. 12 will be discussed with reference to theflowchart of FIG. 13. The area determination section 112 detects thepresence or absence of a character area in input video 2 provided bydecoding signal received by the second tuner 109 b of the video inputsection 104 by the decoder 110 b (step S201). To detect a character areaby the area determination section 112, a widely known conventional artsuch as a method of extracting a character portion using the luminancedifference is used.

If the area determination section 112 detects a character portion (Y atstep S202), it determines that “character area exists” and holds thestate internally (step S203). On the other hand, if the areadetermination section 112 does not detect a character portion (N at stepS202), the area determination section 112 determines that “characterarea does not exist” and terminates the processing (step S204).

If a character portion exists (Y at step S202), the area determinationsection 112 inquires the supplemental information storing section 113whether or not a character area table for input video exists (stepS205). If response from the supplemental information storing section 113is “character area table does not exist” (N at step S205), the areadetermination section 112 requests the supplemental information storingsection 113 to create a new character area table (step S206). On theother hand, response from the supplemental information storing section113 is “character area table exists” (Y at step S205), the areadetermination section 112 makes a comparison between the extractedcharacter string and the previous character string and updates thecharacter area table of the supplemental information storing section 113(S207).

For the input video 2 shown in FIG. 14, the area determination section112 determines and extracts the following portions as character areas.The extracted character areas are, for example, “S2 (1401),” “B3(1402),” and “O2 (1403)” shown in the upper left corner of a screen(1400), “9th (1404)” and “2-2 (1404)” shown in the lower right portionof the screen (1400), and “142 Km/h” shown in the upper right corner ofthe screen (1400). As shown in FIG. 15, the area determination section112 stores information concerning the character areas in the characterarea table of the supplemental information storing section 113 (stepS208). Information such as the extracted character string in eachcharacter area, the character size (points), the width (W), the height(H), and the horizontal axis (X) and the vertical axis (Y) indicatingthe position in the screen is recorded in the character area table.

Further, the area determination section 112 detects an object which isnot a character, such as a symbol or mark displayed on the screen (1400)in FIG. 14 and the screen (1800) in FIG. 18 as an object area. Forexample, detected object areas are a symbol (1407/1807) indicating adiamond of a baseball in the upper left portion of the screen(1400/1800), logs (1408/1808 and 1409/1809) indicating the teams ofbaseball in the lower right portion of the screen (1400/1800), a pitcher(1410/1810) standing on the mound in the lower left portion of thescreen (1400/1800), an umpire and a batter (1411/1811 and 1412/1812) inthe upper right portion of the screen (1400/1800), and the like.

If the area determination section 112 detects an object area (Y at stepS209), it inquires the supplemental information storing section 113whether or not an object area table for input video exists (step S209).If response from the supplemental information storing section 113 is“object area table does not exist” (N at step S209), the areadetermination section 112 requests the supplemental information storingsection 113 to create a new object area table (step S210). On the otherhand, response from the supplemental information storing section 113 is“object area table exists” (Y at step S209), the area determinationsection 112 compares the extracted object with the previous object andupdates the object area table of the supplemental information storingsection 113 (S211).

The area determination section 112 records the object area table asshown in FIG. 16, for example, in the supplemental information storingsection 113 (S212). The object ID, the width (W) and the height (H) ofeach object area, and the horizontal axis (X) and the vertical axis (Y)indicating the position in the screen are recorded in the object areatable.

The details of “character area group definition processing” shown atstep 213 in FIG. 13 and “supplemental information definition processing”shown at step S214 will be described later with reference to FIGS. 17 to21. FIG. 17 is a flowchart to show the character area group definitionprocessing shown at step 213 in the flowchart of FIG. 13. FIG. 18 is adrawing of associating the character areas and the object areas of thescreen shown in FIG. 14 with the IDs (T.* and Oj.*) managed in thecharacter area table and the object area table held by the supplementalinformation storing section 113. FIG. 19 is a drawing to show acharacter area group table held by the supplemental information storingsection 113. FIG. 20 is a drawing to a superposition state detectionpattern. FIG. 21 is a drawing to show an example of a margin width usedin internal processing.

The “character area group” means a group to handle overlap of aplurality of extracted character areas as one information unit. In thefirst embodiment, “T.1/T.2/T.3” and “T.4/T.5” are determined overlap and“T.6” does not overlap any other character string. Definition of thethree character area groups will be discussed below:

First, the area determination section 112 inquires the supplementalinformation storing section 113 whether or not a character area grouptable for input video exists. If response from the supplementalinformation storing section 113 is “character area group table does notexist” (N at step S301), the area determination section 112 requests thesupplemental information storing section 113 to create a new characterarea group table (step S302). On the other hand, response from thesupplemental information storing section 113 is “character area grouptable exists” (Y at step S301), the area determination section 112references the character area group table (FIG. 19) held by thesupplemental information storing section 113.

The area determination section 112 references the character area tableshown in FIG. 15 held by the supplemental information storing section113 and adds a superposition detection margin to all character areas(T.*) to detect a superposition state (step S303). In the embodiment,the margin width (pixel units) changing in response to the scalingfactor of input video, for example, as shown in FIG. 21 is preset. Forexample, assuming that when the scaling factor of the input video is25%, the margin width is 10 pixels, superposition detection margin iscalculated according to the following calculation expression:

Width (W)=width (W)+left margin (10)+right margin (10)

Height (H)=height (H)+upper margin (10)+lower margin (10)

Horizontal axis (X)=horizontal axis (X)−left margin (10)

Vertical axis (Y)=Vertical axis (Y)−upper margin (10)

FIG. 20 is a drawing to show a pattern when superposition state of arean and area n+1. Four sides of “a” to “d” exist in area n and thesuperposition state with area n+1 to be checked need to be checked forall the four sides. However, in the description to follow, to simplifythe description, only “overlap of the lower side of T.1 (1801) and theupper side of T.2 (1802)” of “S2 (1401) and “B3 (1402)” extracted frominput video 2 shown in FIG. 14, namely, “T.1 (1801)” and “T.2 (1802)”shown in the screen (1800) in FIG. 18 will be discussed. Only portionswhere apex T.1 c and apex T.1 d of T.1 (area n) in FIG. 20 overlap apexT.2 a and apex T.2 d of T.2 (area n+1) will be discussed and otheroverlap portions will not be discussed. Algorithm for detecting overlapof dispersed objects on a two-dimensional plane is not limited to thatin the first embodiment and any other method may be used.

The area determination section 112 detects superposition using thefollowing calculation expressions for the character area after a marginis added to each of the lower side of T.1 and the upper side of T.2 tobe checked at present:

T.n(Xc)=T.n(X)+T.n(W)

T.n(Xd)=T.n(X)

ΔT.n(Y)={T.n(Y)+T.n(H)}−T.m(Y)

In the expressions, it is assumed that m=n+1 and T.n indicates T.1 andT.m indicates T.2. It is assumed that “Xc” is assignment of a numericvalue of the horizontal axis (X) at apex c of T.1, “Xd” is assignment ofa numeric value of the horizontal axis (X) at apex d of T.1, and “T.m(Y)” is assignment of a numeric value of the vertical axis (Y) at apex a(or b) of T.2 (step S304).

Next, the area determination section 112 uses the followingdetermination expression to determine overlap of the lower side of T.1and the upper side of T.2:

{T.m(X)≦T.n(Xc)≦T.m(X)+T.m(W) or T.m(X)≦T.n(Xd)≦T.m(X)+T.m(W)}

and

0≦ΔT.n(Y)≦margin width threshold value (10)

The expression means that either of “apex c of T.1 exists between apexesa and b of T2” and “apex d of T.1 exists between apexes a and b of T2”and “the difference between the positions of the lower side of T.1 andthe upper side of T.2 indicated by ΔT.n (Y) is equal to or more than 0or is equal to or less than predetermined margin width threshold value.”If the expression is satisfied, the area determination section 112determines that the lower side of T.1 and the upper side of T.2 overlap(step S305).

If it is determined that the characters to be checked overlap (Y at step305), the area determination section 112 references the character areagroup table of the supplemental information storing section 113. If thearea determination section 112 determines that “character area ID (T.n)exists in previous superposition ID” or “character area ID (T.n) existsin most recent superposition ID” (Y at step S306), the areadetermination section 112 references existing character area group andrecords the character area IDs (T.n and T.m) in the most recentsuperposition ID (step S307). If the area determination section 112determines that “character area ID (T.n) exists in neither ofsuperposition IDs” (N at step S306), the area determination section 112defines a new character area group and records character area IDs (T.nand T.m) in the most recent superposition ID (step S308).

If the area determination section 112 determines that the determinationexpression is not satisfied (N at step S305), the area determinationsection 112 determines that the lower side of T.1 and the upper side ofT.2 do not overlap, and references the character area group table heldby the supplemental information storing section 113. If the areadetermination section 112 determines that “character area ID (T.n) isrecorded in the previous superposition ID” (Y at step 309), the areadetermination section 112 references existing character area group andrecords only the character area ID (T.n) in the most recentsuperposition ID (step S310). On the other hand, if the areadetermination section 112 determines that “character area ID (T.n) isnot recorded in the previous superposition ID” (N at step 309), the areadetermination section 112 defines a new character area group and recordsonly the character area ID (T.n) in the most recent superposition ID(step S311).

Thus, the area determination section 112 changes the character area(T.m) to be calculated in the superposition state according to acalculation expression of (m=m+1), etc., for the character area (T.n) tobe checked (step 312). If calculation target remains, the processreturns to step S304 (N at step S313). After the superposition state isconfirmed for all calculation targets (T.m) (Y at step S313), the areadetermination section 112 confirms the superposition state for the nextcharacter area to be checked (T.n+1) (step S314 and N at S315). Last,the area determination section 112 confirms the superposition state ofall character areas for input video (Y at step S315). Then, the areadetermination section 112 calculates and records the smallest charactersize (pt: Point units) in the most recent superposition ID (T.*)referenced by each character area group (Tx.*) of the character areagroup table held by the supplemental information storing section 113(step S316). Further, the area determination section 112 calculates arectangle including each character area in the most recent superpositionID (T.*) referenced by each character area group (Tx.*) and records inthe character area group table (step S317). Thus, the character areagroup table as shown in FIG. 19 is complete.

Next, “supplemental information definition processing” shown at stepS214 in FIG. 13 will be discussed in detail with reference to FIGS. 22to 24. FIG. 22 is a flowchart to show the supplemental informationdefinition processing shown at step S214 in the flowchart of FIG. 13.FIG. 23 shows definition of character area group area and supplementalinformation area in input video 2. FIG. 23 is drawing of associating thecharacter area groups, the object areas, and the supplementalinformation of the screen shown in FIGS. 14 and 18 with the IDs (T.*,Oj.*, and SA-*) managed in the character area group table and the objectarea table held by the supplemental information storing section 113.FIG. 24 is a drawing to show a supplemental information table held bythe supplemental information storing section 113.

The supplemental information definition processing shown at step S214 inFIG. 13 will be discussed. The “supplemental information” is a unit ofinformation with an extracted character area group as a base andinformation overlapping an object area can be collectively handled asone information unit. In the example shown in FIG. 23, “Tx.1 (2301)” ofthe screen (2300) is determined supplemental information 1 (SA-1: 2310)overlapping “Oj.1 (2304)” and “Tx.2 (2302)” is determined supplementalinformation 2 (SA-2: 2311) overlapping “Oj.5 (2308)” and “Oj.6 (2309).”“Tx.3 (2303) is determined supplemental information 3 (SA-3: 2312) notoverlapping any other object area. Definition of the three pieces ofsupplemental information will be discussed below:

First, the area determination section 112 inquires the supplementalinformation storing section 113 whether or not a supplementalinformation table for input video exists. If response from thesupplemental information storing section 113 is “supplementalinformation table does not exist” (N at step S401), the areadetermination section 112 requests the supplemental information storingsection 113 to create a new supplemental information table (step S402).On the other hand, response from the supplemental information storingsection 113 is “character area group table exists” (Y at step S401), thearea determination section 112 references the supplemental informationtable (FIG. 24) held by the supplemental information storing section113.

The area determination section 112 references the character area grouptable and the object area table held by the supplemental informationstoring section 113 and adds a superposition detection margin to allobject area groups (Oj.*) to detect a superposition state (step S403).The margin width (pixel units) changing in response to the scalingfactor of input video, for example, as shown in FIG. 21 is preset. Forexample, assuming that when the scaling factor of the input video is25%, the margin width is 10 pixels, superposition detection margin iscalculated according to the following calculation expression:

Width (W)=width (W)+left margin (10)+right margin (10)

Height (H)=height (H)+upper margin (10)+lower margin (10)

Horizontal axis (X)=horizontal axis (X)−left margin (10)

Vertical axis (Y)=Vertical axis (Y)−upper margin (10)

In the description to follow, to simplify the description, only “overlapof the right side of Tx.1 and the left side of Oj.1” of “Tx.1 (2301)”and “Oj.1 (2304)” shown in FIG. 23 will be discussed as with thecharacter area group extraction processing described above. That is,only portions where right sides Tx.1 b and Tx.1 c of Tx.1 (area n) inFIG. 20 overlap left sides Oj.1 a and Oj.1 d of Oj.1 (area n+1) will bediscussed and other overlap portions will not be discussed.

The area determination section 112 detects superposition using thefollowing calculation expressions for the object area after a margin isadded to each of the right side of Tx.1 and the left side of Oj.1 to bechecked at present:

Tx.n(Yb)=Tx.n(Y)

Tx.n(Yc)=Tx.n(Y)+Tx.n(N)

ΔTx.n(X)={Tx.n(X)+Tx.n(W)}−Oj.m(X)

In the expressions, it is assumed that m=1 and Tx.n indicates Tx.1 andOj.m indicates Oj.1. It is assumed that “Yb” is assignment of a numericvalue of the vertical axis (Y) at apex b of Tx.1, “Yc” is assignment ofa numeric value of the vertical axis (Y) at apex c of Tx.1, and“Oj.m(X)” is assignment of a numeric value of the horizontal axis (X) atapex a (or d) of Oj.1 (step S404).

Next, the area determination section 112 uses the followingdetermination expression to determine overlap of the right side of Tx.1and the left side of Ob.1:

{Oj.m(Y)≦Tx.n(Yb)≦Oj.m(Y)+Oj.m(H) or Oj.m(Y)<Tx.n(Yc)≦Oj.m(Y)+Oj.m(H)}

and

0≦ΔTx.n(X)≦margin width threshold value (10)

The expression means that either of “apex b of Tx.1 exists betweenapexes a and d of Oj.1” and “apex c of Tx.1 exists between apexes a andd of Oj.1” and “the difference between the positions of the right sideof Tx.1 and the left side of Oj.1 indicated by ΔTx.n(X) is equal to ormore than 0 or is equal to or less than predetermined margin widththreshold value.” If the expression is satisfied, the area determinationsection 112 determines that the right side of Tx.1 and the left side ofOj.1 overlap (step S405).

If it is determined that the character area group and object area to bechecked overlap (Y at step 405), the area determination section 112references the supplement information table held by the supplementalinformation storing section 113. If the area determination section 112determines that “the corresponding character area group (Tx.n) isrecorded in parent ID” (Y at step S406), the area determination section112 references existing supplemental information, records the objectarea ID (Oj.m), and sets a superposition flag to “true” (step S407). If“the corresponding character area group (Tx.n) is not recorded in parentID” (N at step S406), the area determination section 112 defines newsupplemental information, records the object area ID (Oj.m), and setsthe superposition flag to “true” (step S408).

If the area determination section 112 determines that the determinationexpression is not satisfied (N at step S405), the area determinationsection 112 determines that the right side of Tx.1 and the left side ofOj.1 do not overlap, and references the supplemental information tableheld by the supplemental information storing section 113. If the areadetermination section 112 determines that “the corresponding characterarea group (Tx.n) is recorded in parent ID” (Y at step S409), the areadetermination section 112 references existing supplemental information,records the object area ID (Oj.m), and sets the superposition flag to“false” (step S410). On the other hand, if the area determinationsection 112 determines that “the corresponding character area group(Tx.n) is not recorded in parent ID” (N at step S409), the areadetermination section 112 defines new supplemental information, recordsthe object area ID (Oj.m), and sets the superposition flag to “false”(step S411).

Thus, the area determination section 112 changes the object area (Oj.m)to be calculated in the superposition state according to a calculationexpression of (m=m+1), etc., for the character area group (T.n) to bechecked (step S412). If calculation target remains, the process returnsto step S404 (N at step S413). After the superposition state isconfirmed for all calculation targets (Oj.m) (Y at step S413), the areadetermination section 112 calculates the occurrence frequency (%) forsupplemental information (SA-n) based on the cumulative total from thefirst creating time of the supplemental information table, and recordsthe occurrence frequency in the supplemental information table (stepS414). Then, the area determination section 112 confirms thesuperposition state in sequence for the next character area group to bechecked (Tx, n+1) (step S415 and N at S416). Last, the areadetermination section 112 confirms the superposition state of allcharacter area groups for input video (Y at step S416). Then, the areadetermination section 112 calculates a rectangle including overlap ofeach character area group (Tx.*) held by the supplemental informationstoring section 113 and superposed object area (Oj.*) and records in thesupplemental information table (step S417). Thus, the character areagroup table as shown in FIG. 24 is complete.

As described above, the area determination section 112 performsprocessing according to the flowcharts shown in FIGS. 12, 13, 17, and22, whereby the supplemental information table shown in FIG. 24 iscreated or updated and is held in the supplemental information storingsection 113. The area determination section 112 repeats the processingevery wait time (Tw seconds), whereby the descriptions of thesupplemental information are updated following change in the inputvideo, so that the screen as shown in FIG. 10 is continuously displayedon the display.

The drawing timing of the supplemental information will be discussedbelow in detail with reference to FIG. 25. FIG. 25 is a drawing to showsupplemental information display determination criterion responsive tothe scaling factor and the screen division mode of the input video. Inthe supplemental information display determination criterion,supplemental information is displayed only if a predetermineddetermination condition is satisfied at step S105 shown in the flowchartof FIG. 12. The predetermined determination condition is that“occurrence frequency (%) recorded in the supplemental information tableheld by the supplemental information storing section 113 is equal to ormore than “the threshold value of occurrence frequency (for example,50%)” and that “character size”×“current input video scaling factor”recorded in the character area group table is equal to or less than“character threshold value (for example, 12 points),” etc.

The former condition in the supplemental information displaydetermination criterion is a condition for determining that a characterstring displayed in one given time period for input video issupplemental information intended by the program providing company of abroadcast station, etc. The condition is provided, whereby informationnot necessarily required when the screen is divided such as a characterstring displayed suddenly and in a single shot like TV commercials orinformation provided in a baseball relay, for example, can be preventedfrom being displayed as supplemental information. In the firstembodiment, the occurrence frequency of supplemental information 3(SA-3: Speed of pitched ball of pitcher in baseball relay) is “12%” andthus the supplemental information display determination section 115determines that supplemental information 3 does not satisfy thecondition as the supplemental information.

The latter condition in the supplemental information displaydetermination criterion is a condition for determining that when inputvideo is scaled down for display, only if it is determined that thedisplayed character string is illegible for the user, it is displayed assupplemental information. For example, to display supplementalinformation, the character size when input video is displayed 100% asthe initial state as shown in FIG. 25 for “two-screen display states Ato D” shown in FIGS. 7( c) to 7(f) is as follows: The character size ofcharacter area group 1 (Tx.1) is “18 points” and the character size ofcharacter area group 2 (Tx.2) is “26 points.” In the “two-screen displaystate A,” the scaling factor of input video 2 is “75%,” the charactersize of character area group 1 (Tx.1) is “about 14 points,” thecharacter size of character area group 2 (Tx.2) is “about 19 points” andexceeds the threshold value (12 points) and thus it is determined thatinformation is legible for the user and supplemental information is notdisplayed (FIG. 7( c)).

Next, before the “two-screen display state B” is reached, the charactersize of character area group 1 (Tx.1) falls below the threshold value(12 points), the supplemental information display determination section115 starts display of supplemental information. In the “two-screendisplay state B,” the character size of character area group 2 (Tx.2) alittle exceeds the threshold value (12 points) and thus onlysupplemental information 1 is displayed (FIG. 7( d)). Further, justafter the “two-screen display state B” is passed through, before the“two-screen display state C” is reached, the character size of characterarea group 2 (Tx.2) falls below the threshold value (12 points) and thusin the “two-screen display state C,” supplemental information 2 is alsodisplayed (FIG. 7( e). Then, on and after the “two-screen display stateD,” both the character sizes of character area groups 1 and 2 fall belowthe threshold value (12 points) and thus supplemental information 1 andsupplemental information 2 are continuously displayed (FIG. 7( f).

In the description, “supplemental information whose occurrence frequencyis “50%” or more is displayed,” but this determination criterion may beable to be changed as desired by the user. In the description, thethreshold value of the character size for determining display is “12points” by way of example, but may be able to be changed as desired bythe user because the character size illegible for the user varies fromone person to another. The threshold value of the character size fordetermining display may be determined in association with the size setin application for displaying a character string such as font sizesetting of a menu screen, etc., managed by the terminal system, fontsize setting of the browser installed in the terminal, or font sizesetting of EPG installed in the terminal (last setup size).

Drawing positions of supplemental information will be discussed below indetail with reference to FIGS. 26 and 27. FIG. 26 is a drawing to showan example of a factor for determining the display location ofsupplemental information. FIG. 27 is a drawing to show an example of thelayout configuration of a display screen.

As shown in FIG. 26, supplemental information 1 (SA-1: 2601),supplemental information 2 (SA-2: 2602), and supplemental information 3(SA-3: 2603) are defined for an input video screen (2600). In the firstembodiment, areas of a first quadrant to a fourth quadrant provided bydividing the screen into four are defined and the supplementalinformation display determination section 115 determines which quadranteach supplemental information belongs to, thereby determined the roughdisplay position of the supplemental information. For example,supplemental information 1 (SA-1) belongs to the second quadrant,supplemental information 2 (SA-2) belongs to the fourth quadrant, andsupplemental information 3 (SA-3) belongs to the first quadrant.

FIG. 27 shows the screen layout configuration of “two-screen displaystate C” shown in FIG. 7( e). The supplemental information displaydetermination section 115 determines the display position ofsupplemental information at step S106 shown in the flowchart of FIG. 12and transfers to the screen configuration management section 103 as thescreen layout configuration. Here, for example, supplemental information1 (SA-1) contained in input image 2 displayed on screen 2 is positionedin the second quadrant as shown in FIG. 26. Thus, in FIG. 27, apredetermined display margin for the boundary line between screens 1 and2 is subtracted and information is left justified in an upper portion ofscreen 2 and is displayed in the actual size. Further, supplementalinformation 2 (SA-2) contained in input video 2 displayed on screen 2 ispositioned in the fourth quadrant as shown in FIG. 26. Thus, in FIG. 27,display margin is subtracted for the right end of the display framerather than the boundary line between screens 1 and 2 and information isright justified in a lower portion of screen 2 and is displayed in theactual size.

In the example, the configuration ratio of screen division is 3:1 andcentering is performed with the boundary line between screens 1 and 2 asthe borderline, but the display method is not limited to it. Forexample, in the display method, a display margin may be provided betweenscreens or a frame may be given to each screen. In the embodiment,supplemental information is displayed in the actual size (namely, formatof cutting out a part of 100% display of input image), but scaling up ordown may be able be selected as desired by the user. Scaling ofsupplemental information may be changed automatically in associationwith the layout configuration of the screen. When the screenconfiguration management section 103 cross-references areas ofsupplemental information associated with input video and held anddisplays supplemental information in input video 2, the screenconfiguration management section 103 may perform processing consideringthe position of a character area group in input video 1. The processingspecifically is processing of changing the position or the display sizeof supplemental information so that supplemental information of inputvideo 2 does not superpose the character area group of input video 1 orprocessing of displaying no supplemental information.

In the embodiment, a program broadcasted from a broadcast station (forexample, a drama or a baseball relay) is taken as an example of inputvideo. However, it is not a broadcast program and may be any channel orany format such as storage program stored by recording, etc., by anotherdevice, video photographed by a person, a video clip created by a thirdparty, or a moving image distributed through a network.

Whether or not supplemental information held by the supplementalinformation storing section 113 of the embodiment is displayed isdetermined by condition determination (step S105) shown in FIG. 12 bythe supplemental information display determination section 115 by way ofexample. However, the supplemental information display determinationsection 115 may add a condition determination in such a manner that itmakes a comparison with text data of a program outline, etc.,accompanying a broadcast and displays supplemental information only ifit matches the contents.

As described above, the video output device 100 of the embodimentdynamically manages the drawing position and the drawing size of atleast one input video and performs processing of automatically scalingup an area where occurrence of crush of a character string is determinedin scaled-down input image as supplemental information. Accordingly, inthe embodiment, the number of operation times of the user required forscale-up display is decreased and information easy to understand isprovided in a legible character size, whereby occasion loss ofinformation recognition by the user can be prevented.

In scaled-down input video, a symbol of a nearby logo mark, etc., whichis not a character string is also automatically scaled up for display assupplemental information in addition to an area where occurrence ofcrush of a character string is determined. Accordingly, in theembodiment, information loss of a symbol such as a logo mark by simpletext scaling up is prevented and accurate supplemental information canbe provided for the user.

When at least one input image is scaled down for display, supplementalinformation is displayed in an unused area other than the scaled-downdisplay screen (for example, portion of a black belt at the two-screendividing time). Accordingly, in the embodiment, main viewed video is notdisturbed and an empty area can be utilized effectively.

When at least one input image is scaled down for display, supplementalinformation is displayed in the position close to the character areawhich becomes the reference source of supplemental information on thescaled-down display screen. Accordingly, in the embodiment, the user candetermine by intuition which information is scaled up for display on thescaled-down screen and information easy to understand can be provided.

When at least one input image is scaled down for display, the area whichbecomes the reference source of supplemental information and scaled-upsupplemental information area are visually associated with each by aguideline on the scaled-down display screen. Accordingly, the user candetermine by intuition which information is scaled up for display on thescaled-down screen and information easy to understand can be provided.

To display supplemental information in the format of superposing video,the display position of the supplemental information is determinedconsidering what position of video a character area exists at.Accordingly, supplemental information or any other video is notsuperposed on the character area on video and hiding of character areadisplay on video can be prevented.

If a plurality of character areas exist in input video, supplementalinformation of the character area determined the smallest in thecharacter area group including the plurality of character areas ispreferentially displayed automatically. Accordingly, in the embodiment,extra operation for the user is not required and the character stringinformation hardest to see, of crushed character information by scalingdown video can be provided as supplemental information.

In the embodiment, only information displayed for a given time inextracted character areas is displayed as supplemental information.Accordingly, in the embodiment, excessive supplemental informationdisplay of instantaneously displayed subtitles, an advertising signdisplayed accidentally on video, or a character string displayedsuddenly and in a single shot in commercials, etc., can be prevented andpossible screen flickering occurrence can be suppressed.

In the embodiment, change in an extracted character string is determinedand change in supplemental information is reported as enhancementrepresentation on the screen. Accordingly, in the embodiment, the userneed not take care of change in supplemental information at all timesand can recognize by intuition a state change in the scaled-down screenby automatically provided visible enhancement display.

In the embodiment, supplemental information matching subsidiaryaccompanying video, of supplemental information extracted from video isdisplayed preferentially. Accordingly, in the embodiment, display of acharacter area not directly involved in the contents of video, such as acharacter string of an advertising sign displayed accidentally on video,etc., can be prevented.

Second Embodiment

The first embodiment shows the case wherein each character area and eachobject area are extracted from input video and if each character areaoverlaps an object area, area is cut out as one supplemental informationand the supplemental information is superposed for display on an emptyarea of the screen layout configuration managed by the screenconfiguration management section 103 in response to the scaling factorof input video. In a second embodiment, the case where a video outputdevice 100 acquires and displays supplemental information from anexternal server through a network 600 in addition to supplementalinformation extracted by area determination section 112 of the videooutput device 100 will be discussed.

In the embodiment, processing of acquiring and displaying supplementalinformation from an external server will be discussed with reference toFIGS. 28 to 30. FIG. 28 is a block diagram to show the video outputdevice of the second embodiment and its relevant devices. In FIG. 28,the internal configuration of a video input section 104 of the videooutput device 100 is not shown. FIG. 29 is a drawing to show screenconfiguration data provided by a supplemental information providingdevice. FIG. 30 is a drawing to show a display example of supplementalinformation based on the screen configuration data.

The video output device 100 of the embodiment has a similarconfiguration to that of the video output device 100 of the firstembodiment of the first embodiment. The second embodiment in FIG. 28differs from the first embodiment shown in FIG. 6 mainly in that acommunication section 109 d connected to the network 600 is connected toa screen configuration management section 103 and that a supplementalinformation providing device 800 is added as an external serverconnected to the network 600. The configuration other than the above issimilar to that in FIG. 6 and therefore parts similar to those in FIG. 6are denoted by the same reference numerals in FIG. 28 and will not bediscussed again.

The screen configuration management section 103 of the embodimentrequests the supplemental information providing device 800 associatedwith input video displayed on an output section 108 through a combiningsection 107 to acquire screen configuration data through thecommunication section 109 d as required. The supplemental informationproviding device 800 returns screen configuration data as shown in FIG.29 describing the display position of supplemental information, etc., inthe screen configuration of input video being viewed in the video outputdevice 100 in response to the request from the video output device 100.The screen configuration management section 103 analyzes the screenconfiguration data acquired through the communication section 109 d anddetermines which area supplemental information is displayed in for inputvideo.

For example, for the input video 2 (baseball relay) used in thedescription of the first embodiment, three pieces of supplementalinformation 1 to 3 are defined as shown in FIG. 29. As supplementalinformation 1 to 3 in the second embodiment batter score (RP-1),progress information (RP-2), and speed of pitched ball (RP-3) exist andinformation of width (W), height (H), horizontal axis (X), and verticalaxis (Y) is described as the display position of each supplementalinformation. Further, as one embodiment, supplemental information URLs(Uniform Resource Locators) of Japanese version and English version areadded. For the supplemental information URL, in addition to informationconcerning the display position of supplemental information, thesupplemental information providing device 800 provides supplementalinformation and the information can be acquired through the network 600.

The case where language setting is selected as Japanese as one ofprofiles that can be selected as desired by the user operating the videooutput device 100 will be discussed. The profile containing the languagesetting is recorded in a record medium that the screen configurationmanagement section 103 or the video output device of the embodiment has.For example, it is assumed that the progress information (RP-2) and thespeed of pitched ball (RP-3) are represented in English. The screenconfiguration management section 103 acquires “supplemental informationof Japanese version” provided by the supplemental information providingdevice 800 using supplemental information URL in accordance with thelanguage setting and superposes it on input video for display assupplemental information. For example, as shown in FIG. 30, the displayareas of supplemental information 1 to 3 (3001 to 3003) intended by thebroadcast station can be determined in a screen (3000) according to thescreen configuration data. The screen configuration management section103 displays the portion displayed as “9th” in essential input video 2as “9 kai” and the portion displayed as “142 Km/h” as “Jisoku 142 Km”according to the image acquired using supplemental information URL.

In the second embodiment, an image is acquired using supplementalinformation URL, but a moving image distributed in streaming or text maybe acquired.

In the second embodiment, the image configuration data acquired from thesupplemental information providing device 800 through the network 600contains outline of supplemental information indicated by the programprovider of a broadcast station, etc., position information of displayarea, and supplemental information URL to supplemental informationprovided by the supplemental information providing device 800 by way ofexample. However, the contained information is not limited to them andrelevant information, URL for accessing relevant information, the secondaudio program channel that can be acquired through the network 600, andthe like may be contained. The user can change setting of thesupplemental information providing device 800 by one of applicationprocessing section 105 of the video output device 100 or a browser,etc., of a personal computer. To change the setting, the displayposition of supplemental information relative to input video or therelative display position of supplemental information to the screen(3000) when input screen is scaled down (or screen division of inputvideo is performed) may be changed as supplemental information displayspecification area.

As described above, in the embodiment, supplemental information isselected and displayed in accordance with the profile of the user fromamong different types of supplemental information. Thus, in the videooutput device in which Japanese is set as language setting, supplementalinformation of Japanese version is displayed. The screen configurationmanagement section 103 may translate supplemental information into adifferent language. For example, the screen configuration managementsection 103 translates text extracted as English into Japanese fordisplay in accordance with an English-Japanese dictionary. Thus, in theembodiment, supplemental information can be displayed in response to thelanguage setting selected by the user.

The video output device may extract and display video which becomessupplemental information using a supplemental information area containedin supplemental information area pattern data acquired from the outsideof the video output device. The supplemental information area patterndata is data indicating the position of the supplemental informationarea in input video. In this case, processing load on the video outputdevice required for calculating the supplemental information area islightened and the video output device can reliably determine thesupplemental information area intended by the video producer.

The video output device may display supplemental information inaccordance with supplemental information display specification areacontained in the screen configuration data acquired from the outside ofthe video output device. In this case, processing load on the videooutput device required for calculating the display position and the sizeof supplemental information is lightened and the video output device canreliably display the supplemental information in the specification areaintended by the user.

The invention is not limited to the first or second embodiment describedabove. The following cases are also included in the invention:

(1) Each device described above is specifically a computer system madeup of a microprocessor, ROM, RAM, a hard disk unit, a display unit, akeyboard, a mouse, etc. A computer program is stored in the RAM or thehardware disk unit. The microprocessor operates in accordance with thecomputer program, whereby each device accomplishes the function. Thecomputer program is made up of instruction codes indicating commands forthe computer to accomplish a predetermined function.

(2) Some or all of the components making up each device described abovemay be implemented as one system LSI (Large Scale Integration). Thesystem LSI is a supermultiple function LSI manufactured by integrating aplurality of components on one chip and specifically is a computersystem made up of a microprocessor, ROM, RAM, etc. A computer program isstored in the RAM. The microprocessor operates in accordance with thecomputer program, whereby the system LSI accomplishes the function.

(3) Some or all of the components making up each device described abovemay be implemented as an IC card or a single module that can be attachedto or detached from each device. The IC card or the module is a computersystem made up of a microprocessor, ROM, RAM, etc. The IC card or themodule may contain the above-mentioned supermultiple function LSI. Themicroprocessor operates in accordance with the computer program, wherebythe IC card or the module accomplishes the function.

(4) The invention may be the methods described above. The methods may becomputer programs executed by a computer.

The invention may be recorded in a computer-readable record medium ofthe computer program or the digital signals, for example, a flexibledisk, a hard disk, an optical disk, semiconductor memory, etc.

The invention may transmit the computer program via a telecommunicationline, a wireless or wired communication line, a network typified by theInternet, data broadcast, etc.

The invention may be a computer system including a microprocessor andmemory and the memory may store the computer program and themicroprocessor may operate in accordance with the computer program.

The program or the digital signals are recorded in the record medium andthe record medium is moved or the program or the digital signals aremoved via the network, etc., whereby the program or the signals may beexecuted by another independent computer system.

(5) The embodiments and the modified examples may be combined.

While the invention has been described in detail with reference to thespecific embodiments, it will be obvious to those skilled in the artthat various changes and modifications can be made without departingfrom the spirit and the scope of the invention.

This application is based on Japanese Patent Application (No.2008-264004) filed on Oct. 10, 2008, which is incorporated herein byreference.

INDUSTRIAL APPLICABILITY

The video output device according to the invention can be applied to adigital TV receiver, a digital video disk recorder, a set top box, aprojector, an external monitor, a personal computer for outputting videoto the outside, a mobile device having a video display output function,etc.

DESCRIPTION OF REFERENCE NUMERALS

-   100 Video output device-   101 Input section-   102 Terminal control section-   103 Screen configuration management section-   104 Video input section-   105 Application processing section-   106 Screen generation section-   107 Combining section-   108 Output section-   109 a First tuner-   109 b Second tuner-   109 c External input section-   109 d Communication section-   110 Decoder-   111 Processing section-   112 Area determination section-   113 Supplemental information storing section-   115 Supplemental information display determination section-   200 Antenna-   300 Broadcast station-   400 Sending station-   500 Video providing device A-   600 Network-   700 Video providing device B-   800 Supplemental information providing device

1-17. (canceled) 18: A video output device comprising: a screenconfiguration management section that manages a placement or a scalingof a screen for displaying at least one video; an area determinationsection that determines a character area in an input video; asupplemental information storing section that determines whether or notan object area exists in the proximity of an at least one character areadetermined by the area determination section, cuts out a video on asupplemental information area when the object area exists, and storesthe video on the supplemental information area as supplementalinformation including the object area; a supplemental informationdisplay determination section that determines whether or not thesupplemental information stored by the supplemental information storingsection is displayed in addition to an input video based on a screenconfiguration managed by the screen configuration management section;and an output section that outputs the input video and the supplementalinformation in accordance with commands from the screen configurationmanagement section and the supplemental information displaydetermination section, wherein the supplemental information displaydetermination section determines whether the supplemental information isdisplayed based on a size of a character included in a character area inthe input video which changes in response to the scaling managed by thescreen configuration management section. 19: The video output deviceaccording to claim 18, wherein the supplemental information displaydetermination section determines that an area other than the area fordisplaying the at least one video is a display position of thesupplemental information. 20: The video output device according to claim18, wherein the supplemental information display determination sectiondetermines a display position and a display size of the supplementalinformation in accordance with a position of the supplementalinformation area in the input video including the supplementalinformation, a relative position to another screen, and a predeterminedmargin width. 21: The video output device according to claim 18, whereinwhen the supplemental information is displayed, the supplementalinformation display determination section performs a processing so as todisplay in the input video a frame indicating the supplementalinformation area in the input video including the supplementalinformation and display a guide indicating a relationship between thesupplemental information and the frame. 22: The video output deviceaccording to claim 18, wherein the supplemental information displaydetermination section determines a display position and a display sizeof the supplemental information so as not to overlap other supplementalinformation obtained by cutting out a video on a supplementalinformation area including another character area extracted by the areadetermination section. 23: The video output device according to claim18, wherein the supplemental information display determination sectionpreferentially displays supplemental information obtained by cutting outa video on a supplemental information area including a character areawhose character size is determined as the smallest in the at least onecharacter area extracted by the area determination section. 24: Thevideo output device according to claim 18, wherein the supplementalinformation display determination section stores a determination resultof the area determination section into the supplemental informationstoring section and determines preferentially displaying thesupplemental information area existing for a predetermined time or morein the input video among supplemental information areas including the atleast one character area or an object area extracted by the areadetermination section. 25: The video output device according to claim18, wherein the supplemental information storing section stores acharacter string extracted from the character area in the supplementalinformation; and wherein when the character string stored in thesupplemental information storing section differs from the most recentextracted character string from the same character area, thesupplemental information display determination section performs aprocessing so as to emphasize the supplemental information for display.26: The video output device according to claim 18, wherein thesupplemental information storing section stores a character stringextracted from the character area in the supplemental information inaddition to the supplemental information; and wherein the supplementalinformation display determination section performs a processing so as toconvert the character string in accordance with a profile of a user ofthe video output device and display the converted character strings. 27:The video output device according to claim 18, comprising: acommunication section that receives screen configuration data indicatinga placement and a size of a screen for displaying at least one videofrom an external server, wherein the screen configuration managementsection manages the screen configuration in accordance with the screenconfiguration data acquired from the external server by thecommunication section; and wherein the supplemental information displaydetermination section performs a processing so as to display thesupplemental information in the area specified by the screenconfiguration data. 28: The video output device according to claim 27,wherein the supplemental information display determination sectionperforms a processing so as to display the supplemental information inthe area specified by the screen configuration data based on the size ofthe character included in the character area in the input video whichchanges in response to the scaling managed by the screen configurationmanagement section. 29: The video output device according to claim 18,comprising: a communication section that receives information indicatinga position of the supplemental information area in input video from anexternal server, wherein a video which serves as supplementalinformation is extracted and displayed by using the supplementalinformation area contained in the information acquired from the externalserver by the communication section. 30: A video output devicecomprising: a screen configuration management section that manages aplacement or a scaling of a screen for displaying at least one video; anarea determination section that determines a character area in an inputvideo; a communication section that acquires a character string includedin at least one character area determined by the area determinationsection as supplemental information from an external server; asupplemental information display determination section that determineswhether or not the supplemental information acquired by thecommunication section is displayed in addition to an input video basedon the screen configuration managed by the screen configurationmanagement section; and an output section that outputs the input videoand the supplemental information in accordance with commands from thescreen configuration management section and the supplemental informationdisplay determination section, wherein when a size of a characterincluded in the at least one character area determined by the areadetermination section is smaller than a threshold value, thesupplemental information display determination section displays thesupplemental information. 31: The video output device according to claim30, wherein the communication section receives screen configuration dataindicating a placement and a size of a screen for displaying at leastone video from an external server, wherein the screen configurationmanagement section manages the screen configuration in accordance withthe screen configuration data acquired from the external server by thecommunication section; and wherein the supplemental information displaydetermination section performs a processing so as to display thesupplemental information in an area specified by the screenconfiguration data. 32: A video output method, comprising: determining acharacter area in an input video; determining whether or not an objectarea exists in the proximity of an at least one character area, cuttingout a video on a supplemental information area when the object areaexists, and storing the video on the supplemental information area assupplemental information including the object area; determining whetheror not to display the supplemental information in addition to an inputvideo based on the screen configuration for defining a placement or ascaling of a screen for displaying at least one video; and outputtingthe input video and the supplemental information in accordance with thescreen configuration and the determination result, wherein it isdetermined that the supplemental information is displayed based on asize of a character included in a character area in the input video. 33:A video output method, comprising: determining a character area in aninput video; acquiring a character string included in at least onecharacter area as supplemental information from an external server;determining whether or not to display the supplemental informationacquired from the external server in addition to an input video based onthe screen configuration for defining a placement or a scaling of ascreen for displaying at least one video; and outputting the input videoand the supplemental information in accordance with the screenconfiguration and the determination result, wherein when a size of acharacter included in the at least one character area is smaller than athreshold value, it is determined that the supplemental information isdisplayed.